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Abstract : 

We  developed  several  nonparametric  modeling  techniques  as  well  as 
nonlinear  controller  design  techniques.  We  successfully  implementing 
these  techniques  on  a  robot  arm.  We  used  a  locally  weighted  modeling 
approach  as  the  basis  of  our  nonparametric  modeling  technique. 

Locally  weighted  modeling  avoids  negative  interference  by  retaining 
the  original  training  data,  so  the  approach  is  adaptive  to  changing 
data  distributions.  We  used  sophisticated  representations  of  high 
dimensional  sub-manifolds  to  enable  dynamic  programming  in  higher 
dimensional  spaces  than  are  currently  possible. 
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1  Objectives 

There  were  two  major  objectives  in  this  project:  to  develop  a  practical  nonparametric  mod¬ 
eling  approach  and  a  complementary  nonlinear  control  design  approach  for  high  performance 
maneuvers.  The  desired  characteristics  of  the  nonparametric  modeling  approach  were: 

•  The  approach  can  be  applied  automatically  or  with  minimal  human  supervision. 

•  The  approach  can  be  applied  for  online  system  identification. 

•  The  approach  does  not  suffer  from  interference. 

•  The  approach  reduces  the  cost  of  nonlinear  modeling. 

The  desired  characteristics  of  the  nonlinear  control  design  approach  were: 

•  The  approach  can  be  applied  automatically  or  with  minimal  human  supervision. 

•  The  controller  designs  can  be  based  on  the  identified  nonparametric  models. 

•  The  approach  reduces  the  cost  of  nonlinear  controller  design. 

The  approaches  developed  were  to  be  explored  theoretically,  in  simulation,  and  in  an  actual 
implementation. 

2  State  Of  The  Art 

Research  on  nonlinear  modeling  techniques  advances  the  state  of  the  art  in  nonlinear  and 
learning  control.  A  major  challenge  in  the  control  of  complex  vehicles  is  dealing  with  the 
nonlinear  dynamics  of  the  vehicle.  Learning  algorithms  are  beginning  to  be  applied  for  non¬ 
linear  control,  with  the  most  promising  applications  coming  in  situations  classically  handled 
with  gain  scheduling  or  with  nonlinear  inversion.  These  “classical”  approaches  exploit  de¬ 
tailed  mathematical  models  of  the  nonlinear  dynamics;  the  promise  of  learning  algorithms  is 
that  high  fidelity  nonlinear  control  might  be  implemented  even  when  suitable  first  principle 
nonlinear  models  of  the  dynamics  are  lacking  or  expensive  to  obtain. 

Using  a  nonlinear  parametric  model  typically  requires  an  assumption  that  the  model 
structure  is  correct.  This  is  rarely  the  case,  and  motivates  the  search  for  modeling  techniques 
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that  can  correct  structural  modeling  errors.  In  cases  where  nonlinear  models  based  on  fixed 
structure  neural  networks  have  been  shown  to  be  able  to  approximate  any  function,  large 
amounts  of  resources  (exponential  in  the  dimensionality  of  the  state)  have  been  required. 
Another  approach  is  to  add  new  resources  as  needed,  by  adding  new  parameters  or  terms 
to  the  model  structure.  Such  techniques  are  being  explored  in  the  field  of  neural  networks 
where  new  neurons  are  added  to  an  existing  net,  and  in  statistics  where  approaches  such  as 
additive  regression  and  projection  pursuit  add  new  terms  to  the  model.  These  techniques, 
although  promising,  have  not  been  adequately  evaluated  in  adaptive  control  applications. 

3  Approach 

3.1  Nonparametric  Modeling  Based  On  A  Locally  Weighted  Cri¬ 
terion 

We  have  chosen  to  explore  a  different  approach  that  avoids  difficult  issues  such  as  choosing 
an  appropriate  model  structure  in  advance  of  collecting  the  data  [4,7].  The  locally  weighted 
modeling  approach  simply  stores  data,  which  in  a  typical  application  would  be  the  modeling 
errors  of  a  parametric  model  based  on  knowledge  of  the  plant.  When  a  query  is  made  to  the 
parametric  model,  a  new  local  correction  model  is  formed  using  a  locally  weighted  training 
criterion: 

CM  =  Y.  [(/(^m/^)  -  (1) 

i 

the  fth  training  data  point  has  an  input  vector  x,-  and  an  output  y,-,  /()  is  a  model  structure 
with  a  parameter  vector  /?,  K{)  is  the  weighting  or  kernel  function  such  as  a  one  dimensional 
Gaussian,  and  (/(x,-,  q)  is  the  distance  between  the  data  point  Xj  and  the  query  q.  Using  this 
training  criterion,  /(x,/3)  becomes  a  local  model,  and  can  have  a  different  set  of  parameters 
(3  for  each  query  point  q  [7]. 

The  locally  weighted  modeling  approach  has  excellent  asymptotic  properties,  but  little  is 
known  about  how  well  it  will  perform  on  finite  data  sets.  We  have  applied  it  to  the  control 
of  robots  with  excellent  results  [1,2,8]. 

Linear  systems  require  persistent  excitation  for  accurate  system  identification.  With 
nonlinear  systems  and  an  approximately  correct  model  structure,  the  parameters  identified 
depend  on  the  data  distribution.  Negative  interference  is  the  loss  of  the  ability  to  fit  a 
previous  data  distribution  because  of  training  on  a  new  data  distribution.  Locally  weighted 
modeling  avoids  negative  interference  by  retaining  the  original  training  data,  so  the  approach 
is  adaptive  to  changing  data  distributions  [7]. 

3.2  Nonlinear  Controller  Design  Based  On  Dynamic  Program¬ 
ming 

Dynamic  programming  provides  a  methodology  to  design  controllers  and  estimators  for  non¬ 
linear  systems.  However,  general  dynamic  programming  is  intractable.  We  explored  using 
dynamic  programming  in  tubes  around  the  trajectory  of  a  maneuver,  and  in  bubbles  around 
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a  goal  state.  We  used  sophisticated  representations  of  high  dimensional  sub-manifolds  to  en¬ 
able  dynamic  programming  in  higher  dimensional  spaces  than  are  currently  possible  [3,9,10]. 

4  Results 

We  developed  several  nonparametric  modeling  techniques  [1,4,6, 7]  as  well  as  controller  design 
techniques  [2,3,8,9,10]  for  high  performance  control  of  maneuvers.  We  explored  these  schemes 
in  simulation  as  well  as  implementing  them  on  a  nonlinear  testbed.  The  nonlinear  testbed 
was  a  complex  seven  degree  of  freedom  robot  arm.  These  implementations  worked  well, 
demonstrated  real-time  learning,  and  avoided  negative  interference  when  learning  different 
maneuvers.  The  techniques  were  capable  of  learning  maneuvers  both  autonomously  and  from 
demonstrations  by  experts.  These  implementations  demonstrated  the  potential  of  control 
design  techniques  based  on  learned  local  nonparametric  models. 

We  developed  and  implemented  techniques  to  improve  the  performance  of  locally  weighted 
modeling  in  the  areas  of: 

•  Modeling  the  bias  and  variance  of  locally  weighted  model  predictions.  Our  predictions 
are  linear  in  the  data,  leading  to  straightforward  estimates  of  prediction  bias,  variance, 
and  confidence  intervals. 

•  Globally  optimizing  fit  parameters  such  as  distance  metrics,  ridge  regression  parame¬ 
ters,  outlier  rejection  thresholds,  and  weighting  function  parameters. 

•  Identifying  and  eliminating  locally  irrelevant  terms  in  the  model. 

•  Locally  optimizing  fit  parameters  such  as  a  smoothing  parameter  or  bandwidth. 

•  Faster  implementations  that  also  use  less  memory  based  on  using  “receptive  fields”, 
each  of  which  maintains  a  local  model. 

4.1  Air  Force  Benefits 

Success  in  this  research  can  have  several  practical  consequences.  Learning  systems  can  allow 
us  to  increase  the  performance  range  of  a  given  vehicle.  Learning  systems  can  optimize 
performance,  improving  efficiency,  range,  and  agility.  Terrain  may  be  followed  more  closely, 
and  better  pursuit  and  evasive  maneuvers  may  be  possible.  Learning  systems  may  also  allow 
us  to  make  use  of  less  costly  components  in  manned  and  unmanned  vehicles,  and  allow  the 
use  of  less  expensive  instrumentation  and  manufacturing  processes  to  produce  parts  and 
vehicles  with  less  exacting  tolerances.  Learning  can  correct  for  component  inaccuracies  as 
long  as  each  component  is  individually  repeatable.  Learning  systems  may  make  complex 
manned  and  unmanned  vehicles  easier  to  fly,  and  easier  to  train  pilots  for.  Specific  pilot 
training  modes  can  be  developed.  Learning  systems  may  make  unmanned  vehicles  usable 
for  a  wider  range  of  missions  and  ground  controller  skills,  and  ground  crew  requirements  for 
UAVs  may  be  reduced.  The  research  can  lead  to  less  expensive  design  processes  for  control 
systems. 
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5  Future  Work 


We  have  shown  that  our  nonparametric  modelling  techniques  are  successful  in  modeling 
nonlinear  systems.  However,  an  open  question  is  how  to  use  these  nonparametric  nonlinear 
models  to  design  robust  control  systems.  We  have  derived  the  bias  and  variance  for  our 
local  modeling  approaches,  and  expressions  for  the  uncertainty  of  local  model  parameters. 
These  techniques  can  be  used  directly  in  robust  controller  design  approaches,  and  in  dynamic 
programming  to  choose  how  to  explore  optimally,  in  addition  to  controlling  optimally. 
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